neural network prediction
How to Adapt Control Barrier Functions? A Learning-Based Approach with Applications to a VTOL Quadplane
Kim, Taekyung, Beard, Randal W., Panagou, Dimitra
In this paper, we present a novel theoretical framework for online adaptation of Control Barrier Function (CBF) parameters, i.e., of the class K functions included in the CBF condition, under input constraints. We introduce the concept of locally validated CBF parameters, which are adapted online to guarantee finite-horizon safety, based on conditions derived from Nagumo's theorem and tangent cone analysis. To identify these parameters online, we integrate a learning-based approach with an uncertainty-aware verification process that account for both epistemic and aleatoric uncertainties inherent in neural network predictions. Our method is demonstrated on a VTOL quadplane model during challenging transition and landing maneuvers, showcasing enhanced performance while maintaining safety.
Enhancing Exploration Efficiency using Uncertainty-Aware Information Prediction
Kim, Seunghwan, Shin, Heejung, Yim, Gaeun, Kim, Changseung, Oh, Hyondong
Autonomous exploration is a crucial aspect of robotics, enabling robots to explore unknown environments and generate maps without prior knowledge. This paper proposes a method to enhance exploration efficiency by integrating neural network-based occupancy grid map prediction with uncertainty-aware Bayesian neural network. Uncertainty from neural network-based occupancy grid map prediction is probabilistically integrated into mutual information for exploration. To demonstrate the effectiveness of the proposed method, we conducted comparative simulations within a frontier exploration framework in a realistic simulator environment against various information metrics. The proposed method showed superior performance in terms of exploration efficiency.
A novel neural network-based approach to derive a geomagnetic baseline for robust characterization of geomagnetic indices at mid-latitude
Kieokaew, Rungployphan, Haberle, Veronika, Marchaudon, Aurélie, Blelly, Pierre-Louis, Chambodut, Aude
Geomagnetic indices derived from ground magnetic measurements characterize the intensity of solar-terrestrial interaction. The \textit{Kp} index derived from multiple magnetic observatories at mid-latitude has commonly been used for space weather operations. Yet, its temporal cadence is low and its intensity scale is crude. To derive a new generation of geomagnetic indices, it is desirable to establish a geomagnetic `baseline' that defines the quiet-level of activity without solar-driven perturbations. We present a new approach for deriving a baseline that represents the time-dependent quiet variations focusing on data from Chambon-la-For\^et, France. Using a filtering technique, the measurements are first decomposed into the above-diurnal variation and the sum of 24h, 12h, 8h, and 6h filters, called the daily variation. Using correlation tools and SHapley Additive exPlanations, we identify parameters that dominantly correlate with the daily variation. Here, we predict the daily `quiet' variation using a long short-term memory neural network trained using at least 11 years of data at 1h cadence. This predicted daily quiet variation is combined with linear extrapolation of the secular trend associated with the intrinsic geomagnetic variability, which dominates the above-diurnal variation, to yield a new geomagnetic baseline. Unlike the existing baselines, our baseline is insensitive to geomagnetic storms. It is thus suitable for defining geomagnetic indices that accurately reflect the intensity of solar-driven perturbations. Our methodology is quick to implement and scalable, making it suitable for real-time operation. Strategies for operational forecasting of our geomagnetic baseline 1 day and 27 days in advance are presented.
Interpreting and generalizing deep learning in physics-based problems with functional linear models
Arzani, Amirhossein, Yuan, Lingxiao, Newell, Pania, Wang, Bei
Although deep learning has achieved remarkable success in various scientific machine learning applications, its black-box nature poses concerns regarding interpretability and generalization capabilities beyond the training data. Interpretability is crucial and often desired in modeling physical systems. Moreover, acquiring extensive datasets that encompass the entire range of input features is challenging in many physics-based learning tasks, leading to increased errors when encountering out-of-distribution (OOD) data. In this work, motivated by the field of functional data analysis (FDA), we propose generalized functional linear models as an interpretable surrogate for a trained deep learning model. We demonstrate that our model could be trained either based on a trained neural network (post-hoc interpretation) or directly from training data (interpretable operator learning). A library of generalized functional linear models with different kernel functions is considered and sparse regression is used to discover an interpretable surrogate model that could be analytically presented. We present test cases in solid mechanics, fluid mechanics, and transport. Our results demonstrate that our model can achieve comparable accuracy to deep learning and can improve OOD generalization while providing more transparency and interpretability. Our study underscores the significance of interpretability in scientific machine learning and showcases the potential of functional linear models as a tool for interpreting and generalizing deep learning.
Neural network analysis of neutron and X-ray reflectivity data: Incorporating prior knowledge for tackling the phase problem
Munteanu, Valentin, Starostin, Vladimir, Greco, Alessandro, Pithan, Linus, Gerlach, Alexander, Hinderhofer, Alexander, Kowarik, Stefan, Schreiber, Frank
Due to the lack of phase information, determining the physical parameters of multilayer thin films from measured neutron and X-ray reflectivity curves is, on a fundamental level, an underdetermined inverse problem. This so-called phase problem poses limitations on standard neural networks, constraining the range and number of considered parameters in previous machine learning solutions. To overcome this, we present an approach that utilizes prior knowledge to regularize the training process over larger parameter spaces. We demonstrate the effectiveness of our method in various scenarios, including multilayer structures with box model parameterization and a physics-inspired special parameterization of the scattering length density profile for a multilayer structure. By leveraging the input of prior knowledge, we can improve the training dynamics and address the underdetermined ("ill-posed") nature of the problem. In contrast to previous methods, our approach scales favorably when increasing the complexity of the inverse problem, working properly even for a 5-layer multilayer model and an N-layer periodic multilayer model with up to 17 open parameters.
Assessing and Improving Neural Network Predictions by the Bootstrap Algorithm
The bootstrap algorithm is a computational intensive procedure to derive nonparametric confidence intervals of statistical estimators in situations where an analytic solution is intractable. It is ap(cid:173) plied to neural networks to estimate the predictive distribution for unseen inputs. The consistency of different bootstrap procedures and their convergence speed is discussed. A small scale simulation experiment shows the applicability of the bootstrap to practical problems and its potential use.
Speeding up deep neural network-based planning of local car maneuvers via efficient B-spline path construction
Kicki, Piotr, Skrzypczyński, Piotr
Abstract-- This paper demonstrates how an efficient representation of the planned path using B-splines, and a construction procedure that takes advantage of the neural network's inductive bias, speed up both the inference and training of a DNN-based motion planner. We build upon our recent work on learning local car maneuvers from past experience using a DNN architecture, introducing a novel B-spline path construction method, making it possible to generate local maneuvers in almost constant time of about 11 ms, respecting a number of constraints imposed by the environment map and the kinematics of a car-like vehicle. I. INTRODUCTION Although autonomous vehicles are researched intensively, Learning through the interaction seems to carry out the most research on motion planning for these vehicles focuses important information to improve the performance of the mostly on managing traffic scenarios and rules [1], [2], paying trained system [7], while it does not impose any upperbounds less attention to the local maneuvers that are necessary on it, unlike supervised learning, which performance to park a car in a crowded city center, to enter a shopping is bounded by the quality of the reference trajectories or mall's garage, or to avoid a collision with another car that human demonstrations. Human drivers perform Although our previously introduced DNN [5] keeps the such local maneuvers intuitively, leveraging the experience path computation time below 50 ms, some emergency maneuvers from similar situations they have encountered in the past. Therefore, we contribute in this seconds) avoiding collisions in dangerous situations, and paper a novel path parametrization and procedure of its satisfying the constraints of a car-like vehicle. A car is nonholonomic, construction, which enables our method to compute yet has a limited steering angle and some physical better paths in an even shorter time in comparison to dimensions, while the planned path should allow control [5] Although these requirements planning function, breaks up with the Markov Decision call for a solution that is rather a reactive behavior than Process formalism used in [5], instead plans the whole a classic planning algorithm, reactive methods [3] rarely maneuver at once.
NSL: Hybrid Interpretable Learning From Noisy Raw Data
Cunnington, Daniel, Russo, Alessandra, Law, Mark, Lobo, Jorge, Kaplan, Lance
Inductive Logic Programming (ILP) systems learn generalised, interpretable rules in a data-efficient manner utilising existing background knowledge. However, current ILP systems require training examples to be specified in a structured logical format. Neural networks learn from unstructured data, although their learned models may be difficult to interpret and are vulnerable to data perturbations at run-time. This paper introduces a hybrid neural-symbolic learning framework, called NSL, that learns interpretable rules from labelled unstructured data. NSL combines pre-trained neural networks for feature extraction with FastLAS, a state-of-the-art ILP system for rule learning under the answer set semantics. Features extracted by the neural components define the structured context of labelled examples and the confidence of the neural predictions determines the level of noise of the examples. Using the scoring function of FastLAS, NSL searches for short, interpretable rules that generalise over such noisy examples. We evaluate our framework on propositional and first-order classification tasks using the MNIST dataset as raw data. Specifically, we demonstrate that NSL is able to learn robust rules from perturbed MNIST data and achieve comparable or superior accuracy when compared to neural network and random forest baselines whilst being more general and interpretable.
[P] Explaining Neural Network Predictions (Open Source) • r/MachineLearning
For attractiveness the moderators were given 3 choices: hot, neutral, and not. We trained the classifier with those 3 categories and the score is a linear combination of the softmax probabilities for each bucket. We tried our best to eliminate bias but i'm sure it still exists based on the human input.
An in-depth look at Google's first Tensor Processing Unit (TPU) Google Cloud Big Data and Machine Learning Blog Google Cloud Platform
There's a common thread that connects Google services such as Google Search, Street View, Google Photos and Google Translate: they all use Google's Tensor Processing Unit, or TPU, to accelerate their neural network computations behind the scenes. We announced the TPU last year and recently followed up with a detailed study of its performance and architecture. In short, we found that the TPU delivered 15–30X higher performance and 30–80X higher performance-per-watt than contemporary CPUs and GPUs. These advantages help many of Google's services run state-of-the-art neural networks at scale and at an affordable cost. In this post, we'll take an in-depth look at the technology inside the Google TPU and discuss how it delivers such outstanding performance.